Common techniques for interaction between humans and machines include hand-operated user interface devices, such as keyboards, buttons, joysticks and pointing devices (e.g. a mouse). Recent developments in eye-gaze tracking systems can determine the line-of-sight (LOS) vector of an individual's eye. This LOS information can be used as a control tool for human machine interaction. There are a number of advantages of using eye-gaze tracking information as a control tool. These advantages include: the intuitive link between the visual system of the eye and the resultant images in the brain; the speed of eye movement relative to moving a hand-operated interaction device (i.e. users typically look at the desired destination of a hand-operated device prior to moving the hand-operated device); and the possibility that eye-gaze tracking techniques may be used by severely disabled individuals.
A number of other applications for eye-gaze tracking systems include, without limitation: psychological and physiological research into the connection between eye movements and perceptual and/or cognitive processes; the analysis of driver awareness; research into the effectiveness of advertising and website layouts; and gaze contingent displays.
A number of prior art references describe various techniques for eye-gaze tracking. These references include:                A. T. Duchowski, Eye Tracking Methodology: Theory and Practice. Springer-Verlag, 2003.        L. Young and D. Sheena, “Methods & designs: survey of eye movement recording methods,” Behav. Res. Methods Instrum., vol. 5, pp. 397-429, 1975.        R. Jacob and K. Karn, The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research. Amsterdam: Elsevier Science, 2003, ch. Eye Tracking in Human-Computer Interaction and Usability Research: Ready to Deliver the Promises (Section Commentary), pp. 573-605.        T. Hutchinson, J. White, W. Martin, K. Reichert, and L. Frey, “Human-computer interaction using eye-gaze input,” Systems, Man and Cybernetics, IEEE Transactions on, vol. 19, no. 6, pp. 1527-1534, November-December 1989.        S.-W. Shih and J. Liu, “A novel approach to 3-d gaze tracking using stereo cameras,” Systems, Man and Cybernetics, Part B, IEEE Transactions on, vol. 34, no. 1, pp. 234-245, February 2004.        D. Beymer and M. Flickner, “Eye gaze tracking using an active stereo head,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, 18-20 Jun. 2003, pp. II-451-8 vol. 2.        C. Hennessey, B. Noureddin, and P. Lawrence, “A single camera eye-gaze tracking system with free head motion,” in Proceedings of the 2006 symposium on Eye tracking research & applications. New York, N.Y., USA: ACM Press, 2006, pp. 87-94.        C. H. Morimoto, A. Amir, M. Flickner, “Detecting Eye Position and Gaze from a Single Camera and 2 Light Sources,” 16th International Conference on Pattern Recognition (ICPR'02)—Volume 4, 2002, p. 40314.        Z. Zhu and Q. Ji, “Eye Gaze Tracking Under Natural Head Movements,” Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005.        E. Guestrin and M. Eizenman, “General theory of remote gaze estimation using the pupil center and corneal reflections,” Biomedical Engineering, IEEE Transactions on, vol. 53, no. 6, pp. 1124-1133, June 2006.        A. T. Duchowski, V. Shivashankaraiah, T. Rawls, A. K. Gramopadhye, B. J. Melloy, and B. Kanki, “Binocular eye tracking in virtual reality for inspection training,” in Proceedings of the 2000 symposium on Eye tracking research & applications. New York, N.Y., USA: ACM Press, 2000, pp. 89-96.        K. Essig, M. Pomplun, and H. Ritter, “Application of a novel neural approach to 3d gaze tracking: Vergence eye-movements in autostereograms,” in Proceedings of the 26thl Meeting of the Cognitive Science Society, K. Forbus, D. Gentner, and T. Regier, Eds., 2004, pp. 357-362.        K. Essig, M. Pomplun, and H. Ritter, “A neural network for 3d gaze recording with binocular eyetrackers,” International Journal of Parallel, Emergent and Distributed Systems (accepted), 2006.        Y.-M. Kwon and K.-W. Jeon, “Gaze computer interaction on stereo display,” in Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology. New York, N.Y., USA: ACM Press, 2006, p. 99.        PCT publication No. WO04/045399 (Elvesjö et al.).        U.S. Pat. No. 4,386,670 (Hutchinson).        U.S. Pat. No. 5,231,674 (Cleveland et al.).        U.S. Pat. No. 5,471,542 (Ragland).        U.S. Pat. No. 5,428,413 (Shindo).        U.S. Pat. No. 6,152,563 (Hutchinson et al.).        U.S. Pat. No. 6,659,611 (Amir et al.).        U.S. Pat. No. 5,481,622 (Gerhardt).        U.S. Pat. No. 6,578,962 (Amir et al.).        
Some of these prior art eye-gaze tracking systems may be used to detect LOS information for one of a user's eyes when the user's eye is fixated at a particular location (referred to as a point-of-gaze (POG)). An eye may be said to be “fixated” on a POG when the POG is imaged onto the eye's fovea and the motion of the eye is stabilized. To the extent that prior art eye-gaze tracking systems are used to estimate a POG using LOS information, the LOS is only used to estimate the POG in two dimensions. For example, where a user's eye is fixated on a two-dimensional monitor screen, the POG may be determined to be the location where the LOS vector intersects with the plane of the monitor screen.
Two-dimensional POG estimation may be satisfactory for interacting with standard two-dimensional human-machine interface environments (e.g. monitor screens). However, there are a number of continually improving three dimensional display technologies, such as volumetric displays and parallax beam splitter displays, for example, which may provide three-dimensional human-machine interface environments—see, for example, M. Halle, “Autostereoscopic displays and computer graphics,” SIGGRAPH Comput. Graph., vol. 31, no. 2, pp. 58-62, 1997. Such three-dimensional user interface environments could provide users with a much richer experience (i.e. more functionality) than existing two-dimensional user interface environments.
For this and other reasons, there is a general desire to provide methods and apparatus for POG estimation in three dimensions.